<!doctype html>
<html lang="en">


<!-- === Header Starts === -->
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

  <title>IDInvert</title>

  <link href="./assets/bootstrap.min.css" rel="stylesheet">
  <link href="./assets/font.css" rel="stylesheet" type="text/css">
  <link href="./assets/style.css" rel="stylesheet" type="text/css">
</head>
<!-- === Header Ends === -->


<body>


<!-- === Home Section Starts === -->
<div class="section">
  <!-- === Title Starts === -->
  <div class="header">
    <div class="logo">
      <a href="https://genforce.github.io/" target="_blank"><img src="./assets/genforce.png"></a>
    </div>
    <div class="title", style="padding-top: 25pt;">
      In-Domain GAN Inversion for Real Image Editing
    </div>
  </div>
  <!-- === Title Ends === -->
  <div class="author">
    <a href="https://zhujiapeng.github.io/" target="_blank">Jiapeng Zhu</a><sup>1</sup>*,&nbsp;
    <a href="http://shenyujun.github.io/" target="_blank">Yujun Shen</a><sup>1</sup>*,&nbsp;
    <a href="https://sites.google.com/site/zhaodeli" target="_blank">Deli Zhao</a><sup>2</sup>,&nbsp;
    <a href="http://bzhou.ie.cuhk.edu.hk" target="_blank">Bolei Zhou</a><sup>1</sup>
  </div>
  <div class="institution">
    <sup>1</sup>The Chinese University of Hong Kong &nbsp; &nbsp; &nbsp; &nbsp;
    <sup>2</sup>Xiaomi AI Lab
  </div>
  <div class="link">
    <a href="https://arxiv.org/pdf/2004.00049.pdf" target="_blank">[Paper]</a>&nbsp;
    <a href="https://github.com/genforce/idinvert" target="_blank">[Code (TensorFlow)]</a>&nbsp;
    <a href="https://github.com/genforce/idinvert_pytorch" target="_blank">[Code (PyTorch)]</a>
    <a href="https://colab.research.google.com/github/genforce/idinvert_pytorch/blob/master/docs/Idinvert.ipynb" target="_blank">[Colab]</a>
  </div>
  <div class="teaser">
    <img src="./assets/teaser.jpg">
  </div>
</div>
<!-- === Home Section Ends === -->


<!-- === Overview Section Starts === -->
<div class="section">
  <div class="title">Overview</div>
  <div class="body">
    In this work, we argue that the GAN inversion task is required
    not only to reconstruct the target image by pixel values,
    but also to keep the inverted code in the <i>semantic domain</i> of the original latent space of well-trained GANs. For this purpose, we propose In-Domain GAN inversion (IDInvert) by
    first training a novel <i>domain-guided</i> encoder which is able to produce in-domain latent code,
    and then performing <i>domain-regularized</i> optimization which involves the encoder as a regularizer to land the
    code inside the latent space when being finetuned.
    The in-domain codes produced by IDInvert enable high-quality real image editing with fixed GAN models.
  </div>
</div>
<!-- === Overview Section Ends === -->


<!-- === Result Section Starts === -->
<div class="section">
  <div class="title">Results</div>
  <div class="body">
    Semantic diffusion results.

    <table width="100%" style="margin: 20pt auto; text-align: center;">
      <tr>
        <td><img src="./assets/teaser_diffusion.gif" width="90%"></td>
      </tr>
    </table>

    Image editing results.

    <table width="100%" style="margin: 20pt auto; text-align: center;">
      <tr>
        <td><img src="./assets/teaser_video.gif" width="90%"></td>
      </tr>
    </table>

    See more results in the following demo video:
    
    <div style="position: relative; padding-top: 50%; margin: 0pt auto; text-align: center;">
      <iframe src="https://www.youtube.com/embed/3v6NHrhuyFY" frameborder=0
              style="position: absolute; top: 0%; left: 2.5%; width: 95%; height: 100%;"
              allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
              allowfullscreen></iframe>
    </div>
    <br>
    This work is featured in <a href="https://www.youtube.com/watch?v=2qMw8sOsNg0">Two Minute Papers</a> Youtube channel as below: 
   
    <div style="position: relative; padding-top: 50%; margin: 0pt auto; text-align: center;">
            <iframe src="https://www.youtube.com/embed/2qMw8sOsNg0" frameborder=0
              style="position: absolute; top: 0%; left: 2.5%; width: 95%; height: 100%;"
              allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
              allowfullscreen></iframe>
    </div>
 
  </div>
</div>
<!-- === Result Section Ends === -->


<!-- === Reference Section Starts === -->
<div class="section">
  <div class="bibtex">BibTeX</div>
<pre>
@inproceedings{zhu2020indomain,
  title     = {In-domain GAN Inversion for Real Image Editing},
  author    = {Zhu, Jiapeng and Shen, Yujun and Zhao, Deli and Zhou, Bolei},
  booktitle = {Proceedings of European Conference on Computer Vision (ECCV)},
  year      = {2020}
}
</pre>

  <div class="ref">Related Work</div>
  <div class="citation">
    <div class="image"><img src="./assets/interfacegan.jpg"></div>
    <div class="comment">
      <a href="https://genforce.github.io/interfacegan/" target="_blank">
        Y. Shen, J. Gu, X. Tang, B. Zhou.
        Interpreting Latent Space of GANs for Semantic Face Editing.
        CVPR 2020.</a><br>
      <b>Comment:</b>
      Proposes a technique for semantic face editing in latent space.
    </div>
  </div>
  <div class="citation">
    <div class="image"><img src="./assets/higan.jpg"></div>
    <div class="comment">
      <a href="https://genforce.github.io/higan/" target="_blank">
        C. Yang, Y. Shen, B. Zhou.
        Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis.
        arXiv preprint arXiv:1911.09267, 2019</a><br>
      <b>Comment:</b>
      Explores the emergent semantic hierarchy in scene synthesis models.
    </div>
  </div>
  <div class="citation">
    <div class="image"><img src="./assets/image2stylegan.jpg"></div>
    <div class="comment">
      <a href="https://arxiv.org/abs/1904.03189" target="_blank">
        R. Abdal, Y. Qin, P. Wonka
        Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
        ICCV 2019.</a><br>
      <b>Comment:</b>
      Explores how to Embed Images into the latent space.
    </div>
  </div>
  <div class="citation">
    <div class="image"><img src="./assets/pix2pix.jpg"></div>
    <div class="comment">
      <a href="https://phillipi.github.io/pix2pix/" target="_blank">
        P. Isola, J.Y. Zhu, T. Zhou, A. A. Efros.
        Image-to-Image Translation with Conditional Adversarial Nets.
        CVPR 2017.</a><br>
      <b>Comment:</b>
      Investigates image-to-image translation using conditional GANs.
    </div>
  </div>
  <div class="citation">
    <div class="image"><img src="./assets/igan.jpg"></div>
    <div class="comment">
      <a href="http://efrosgans.eecs.berkeley.edu/iGAN/" target="_blank">
        J.Y. Zhu, P. Krähenbühl, E. Shechtman, A. A. Efros.
        Generative Visual Manipulation on the Natural Image Manifold.
        ECCV 2016.</a><br>
      <b>Comment:</b>
      Proposes a method for realistic photo manipulation and a system for interactive drawing using GANs.
    </div>
  </div>
  <div class="citation">
    <div class="image"><img src="./assets/mganprior.jpg"></div>
    <div class="comment">
      <a href="https://genforce.github.io/mganprior/" target="_blank">
        J. Gu, Y. Shen, B. Zhou.
        Image Processing Using Multi-Code GAN Prior.
        CVPR 2020.</a><br>
      <b>Comment:</b>
      Employs multiple latent codes to invert a GAN model as prior for real image processing.
    </div>
  </div>
</div>
<!-- === Reference Section Ends === -->


</body>
</html>
